{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Copyright 2019 Google LLC\n",
    "#\n",
    "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
    "# you may not use this file except in compliance with the License.\n",
    "# You may obtain a copy of the License at\n",
    "#\n",
    "#     https://www.apache.org/licenses/LICENSE-2.0\n",
    "#\n",
    "# Unless required by applicable law or agreed to in writing, software\n",
    "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
    "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
    "# See the License for the specific language governing permissions and\n",
    "# limitations under the License."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a target=\"_blank\" href=\"https://colab.research.google.com/github/GoogleCloudPlatform/keras-idiomatic-programmer/blob/master/workshops/Idiomatic Programmer - handbook 1 - Codelab 1.ipynb\">\n",
    "<img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" />Run in Google Colab</a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Idiomatic Programmer - Code Labs \n",
    "\n",
    "## Code Lab #1 - Get Started with a Deep Neural Network (DNN)\n",
    "\n",
    "## Prerequistes:\n",
    "\n",
    "    1. Familiar with Python\n",
    "    2. Completed Handbook 1/Part 1: Neural Networks\n",
    "    \n",
    "## Objectives:\n",
    "\n",
    "    1. Install Keras related packages\n",
    "    2. Create a basic DNN (input layer, hidden layer, output layer)\n",
    "        A. Correctly set the input shape\n",
    "        B. Correctly set the number of nodes\n",
    "        C. Correctly set activation function\n",
    "    3. Review a model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup:\n",
    "\n",
    "Install relevant packages to get started with Keras, and then import them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Install numpy math library\n",
    "!pip install -U numpy\n",
    "# Install Keras framework\n",
    "!pip install -U keras\n",
    "# Install Tensorflow backend\n",
    "!pip install -U tensorflow\n",
    "\n",
    "# Import keras and TF\n",
    "import keras\n",
    "import tensorflow as tf\n",
    "import numpy as np\n",
    "\n",
    "print(np.__version__)              # should be 1.16.4\n",
    "print(tf.__version__)              # should be 1.13.1\n",
    "print(keras.__version__)           # should be 2.2.4"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Basic DNN as Sequential API (long form)\n",
    "\n",
    "Let's use the long form of a Sequential method. That means that we will use the add() method for each step.\n",
    "\n",
    "Below is a nearly complete DNN. You fill in the blanks (replace the ??), make sure it passes the Python interpreter, and then verify it's correctness with the summary output."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras import Sequential\n",
    "from keras.layers import Dense, Activation\n",
    "\n",
    "# define the input shape for a 28x28 grayscale image (like MNIST)\n",
    "# HINT: must explicitly set the number of channels\n",
    "input_shape=( ?? )\n",
    "\n",
    "# Let's start a sequential model\n",
    "model = Sequential()\n",
    "\n",
    "# Let's add the input layer as a dense layer of 128 nodes\n",
    "model.add(Dense(??, input_shape=input_shape))\n",
    "\n",
    "# Add the activation function\n",
    "# HINT: use the best practice convention for a non-output Dense layer\n",
    "model.add(Activation(??))\n",
    "\n",
    "# Add the hidden layer with 512 nodes\n",
    "model.add(Dense(??))\n",
    "model.add(Activation(??))\n",
    "\n",
    "# Add the output layer with 10 nodes\n",
    "model.add(Dense(??))\n",
    "\n",
    "# Add the activation function\n",
    "# HINT: use the best practice for a classifier\n",
    "model.add(Activation(??))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Verify the model architecture using summary method\n",
    "\n",
    "It should look like below:\n",
    "\n",
    "```\n",
    "Layer (type)                 Output Shape              Param #   \n",
    "=================================================================\n",
    "dense_1 (Dense)              (None, 28, 28, 128)       256       \n",
    "_________________________________________________________________\n",
    "activation_1 (Activation)    (None, 28, 28, 128)       0         \n",
    "_________________________________________________________________\n",
    "dense_2 (Dense)              (None, 28, 28, 512)       66048     \n",
    "_________________________________________________________________\n",
    "activation_2 (Activation)    (None, 28, 28, 512)       0         \n",
    "_________________________________________________________________\n",
    "dense_3 (Dense)              (None, 28, 28, 10)        5130      \n",
    "_________________________________________________________________\n",
    "activation_3 (Activation)    (None, 28, 28, 10)        0         \n",
    "=================================================================\n",
    "Total params: 71,434\n",
    "Trainable params: 71,434\n",
    "Non-trainable params: 0\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##  Basic DNN as Sequential API (short form)\n",
    "\n",
    "Let's repeat the above, but use the short form."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras import Sequential\n",
    "from keras.layers import Dense\n",
    "\n",
    "# Let's start a sequential model\n",
    "model = Sequential()\n",
    "\n",
    "# Let's add the input layer as a dense layer of 128 nodes\n",
    "model.add(Dense(128, activation=??, input_shape=(28, 28, 1)))\n",
    "\n",
    "# Add the hidden layer with 512 nodes\n",
    "model.add(Dense(512, activation=??))\n",
    "\n",
    "# Add the output layer with 10 nodes\n",
    "model.add(Dense(10, activation=??))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Verify the model architecture using summary method\n",
    "\n",
    "It should be identical to the previous. But note in this case, summary() does not show the activation functions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Basic DNN as Sequential API (list form)\n",
    "\n",
    "Let's repeat the above, but instead specify the layers as a parameter to the Sequential() object using a list."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras import Sequential\n",
    "from keras.layers import Dense\n",
    "\n",
    "# Let's start a sequential model\n",
    "model = Sequential([ # input layer\n",
    "                     Dense(128, activation='relu', input_shape=(28, 28, 1)),\n",
    "                     # hidden layer\n",
    "                     ??,\n",
    "                     # output layer\n",
    "                     ??\n",
    "                    ])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Verify the model architecture using summary method\n",
    "\n",
    "It should be identical to the previous."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Basic DNN as Functional API \n",
    "\n",
    "Let's repeat the above, but instead specify the layers using the functional API."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras import Model, Input\n",
    "from keras.layers import Dense\n",
    "\n",
    "# Define the input vector\n",
    "inputs = Input(??)\n",
    "\n",
    "# Define the input layer and connect the input vector to it.\n",
    "# HINT: the input vector is where you defined the input shape\n",
    "x = ??(128, activation='relu')(??)\n",
    "\n",
    "# Define the hidden layer and connect the hidden layer to it.\n",
    "x = ??(512, activation='relu')(??)\n",
    "\n",
    "# Define the output layer and connect the hidden layer to it.\n",
    "outputs = ??(10, activation='softmax')(??)\n",
    "\n",
    "# Let's put it together: inputs to outputs\n",
    "model = Model(??, ??)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Verify the model architecture using summary method\n",
    "\n",
    "It should be identical to the previous."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## End of Lab"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
